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ABSTRACT 


The object of this paper was Real Time Facial Expression Recognition (FER] 
has become main area of interest due to its wide applications. Automatic 
Facial expression recognition has drawn the attention of researchers as it has 
many applications. Facial Expression Recognition gives important information 
about emotions of a human being. Many feature selection methods have been 
developed for identification of expressions from still images and real time 
videos. This work gives a detailed review of research works done in the field of 
facial expression identification and various methodologies implemented for 
facial expression recognition. 


INTRODUCTION: 

Our project titled as "Real Time Facial Expression 
Recognition and Imitation" has been undertaken with the 
aim of estimation of facial expression guiding by a program 
for Raspberry Pi to evaluate a person's exact expression or 
expression type. The technique we used for expression type 
estimation contains two main component-type estimation 
divides the types only into rough groups (e.g., angry, disgust, 
fear, happy, sad, surprise, neutral]. 

Studies on human facial emotion recognition have revealed 
that computing models based on regression modeling can 
produce applicable performance. Emotion detection and 
recognition were introduced by researches with human 
observers. Automatic recognition and the study of the facial 
emotional status represent substantial suggestions for the 
way in which a person performs, and these are very helpful 
for detecting, inspection and keeping safe vulnerable 
persons such as patients who experience mental issues, 
persons who endure significant mental pressure, and 
children with less ability to control themselves. With 
emotion recognition ability, machines such as computers, 
robots, toys and game consoles will have the capability to 
perform in such a way as to influence the user in adaptive 
ways relevant for the client's mental condition. This is the 
key knowledge in recently proposed new ideas such as 
emotional computers, emotion-sensing smart phones and 
emotional robots. Most studies have focused on emotional 
symptoms in facial expressions. In recent years, scientists 
have studied effective transition through body language. The 
recognition of entire body gestures is expressively more diff 
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icult, as the shape of the human body has more points of 
freedom than the face, and its overall outline differs 
muscularly through representative motion. 

Most face detection algorithms are designed in the software 
field and have a high detection rate, but they often needs 
several seconds to detect faces in a single image, a 
processing speed that is insufficient for real time 
applications. To overcome this problem we used an easy 
hardware implementation of Raspberry Pi which is the 
fastest method of face detection and estimation of facial 
expression accurately. For the extraction of facial feature of 
human we used camera pi which is an excellent add-on for 
Raspberry Pi Processor to take pictures and record videos 
with the possibility to apply a considerable range of 
configurations and effects. 

An face emotion recognition system comprises of two step 
process i.e. face detection (bounded face] in image followed 
by emotion detection on the detected bounded face. The 
following two techniques are used for respective mentioned 
tasks in face recognition system. 

1. Haar feature-based cascade classifiers: It detects frontal 
face in an image well. It is real time and faster in 
comparison to other face detector. This blog-post uses 
an implementation from Open-CV. 

2. Xception CNN Model: We will train a classification CNN 
model architecture which takes bounded face (48*48 
pixels] as input and predicts probabilities of 7 emotions 
in the output layer. 
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In our day to day life emotions or facial expression are the 
prime factor which are required for communication purpose. 
For humans it is quite easy to detect an emotion but difficult 
for a computer or a machine to do so. Human emotions are 
mainly classified into 7 emotions i.e. Neutral, Happy, Sad, 
Anger, Disgust, Fear and Surprise. This project deals with 4 
emotions they are Neutral, Happy, Sad and Surprise. 

As we are developing the need and importance of automatic 
emotion recognition has increased which supports Human 
Computer Interaction applications. Facial expression defines 
the emotions of an individual which is required for Human 
Computer Interaction (HCI] in this project. Apart from 
Human Computer Interaction the system could be used for 
monitoring medical patients emotional states and stress 
levels. 

Emotion recognition can be done by text, vocal, verbal and 
facial expression. In 1968, Albert Mehrabian pointed out that 
in human to human interaction 7% of communication is 
contributed by verbal cues, 38% is contributed by vocal cues 
and major portion 55% is contributed by facial expressions. 



Disgust Surprise Sad Angry 



Fear Happy Contempt Neutral 



neutral sad anger disgust fear happy surprise 


Methodology - In the proposed method, the objective is to 
develop real time emotion recognition from facial images to 
recognize basic emotions like anger, disgust, happiness, 
surprise and neutral. We have used CMU Multiple database, 
which is a collection of images from 337 subjects with a 
variety of different facial expressions including neutral, 
happiness, surprise, disgust and anger. The subjects include 
235 males and 102 females with different level of 
illuminations and poses. Viola-jones face detection method 
for face detection, Active shape Model (ASM] for extracting 
facial points and AdaBoost classifier have been used for 
developing the emotion recognition software. 



Webcam Raspberry Pi II 

Figl: Real Time Emotion Recognition System 



Figl: Real- time Emotion recognition system 


Raspberry Pi II is a credit card sized computer which has 
system on a chip Broadcom BCM2835. It contains an 
ARM 1176JZFS, with floating point, running at 900 MHz and a 
video core 4 GPU. 

The architecture of proposed system is shown in Fig. 1 and 
explained as follows: The input image in real time is 
captured through webcam and fed to emotion recognition 
software as input. Emotion recognition software is deployed 
in the Raspberry Pi II, which gives classified emotion as 
output. The recognized emotion is displayed in the monitor. 
The operations performed by the software deployed in the 
Raspberry Pi II is shown in Fig. 2. 

The algorithm for real time implementation of emotion 
recognition using Raspberry Pi II is explained as follows Step 
1: Input image is captured through webcam. 

Step 2: Viola-jones [6] face detection technique is used to 
detect the facial image. Viola-jones used Haar wavelet 
concept to develop integral image to detect face. Haar 
features consider the different intensity of values of adjacent 
rectangular region as different area of face has different 
value of intensity from other region. After detection, facial 
image is saved for further processing and non-face area is 
removed. 

Step 3: In image preprocessing, image is cropped according 
to required size and converted in gray image. This cropped 
image is used as input to Sobel filter for smoothing to 
remove the noise. 

Step 4: Feature extraction is based on geometric approach 
for which Active Shape Model [ASM] is used. ASM automatic 
fiducial point location algorithm is applied first to a facial 
expression image, and then Euclidean distances between 
center gravity coordinate and the annotated fiducial points 
coordinates of the face image are calculated. 

In order to extract the discriminate deformable geometric 
information, the system extracts the geometric deformation 
difference features between a person's neutral expression 
and the other basic expressions. In ASM input face shape is 
iteratively deformed to get the shape model. After 
comparison with shape model feature point of input facial 
image is extracted. 


@ IJTSRD | Unique Paper ID - IJTSRD31584 | Volume - 4 | Issue - 4 | May-June 2020 


Page 1523 












































International Journal of Trend in Scientific Research and Development (IJTSRD] @ www.ijtsrd.com elSSN: 2456-6470 


To train the model, frontal images of 5 emotions, 60 subjects 
from CMU MultiPIE database is used where sample images 
from the database are shown in Fig 4. Feature points are 
marked on these images as shown in Fig 5. These points are 
then normalized and formed as a feature vector. This 
operation is repeated for all the subjects and emotions, and 
the feature points forms a single feature vector which is 
given to the classifier for training. 



Ii!»4: Cropped Images funis CMU MULTIPIE ihhhm 



Fig 5: Fiducial points extracted for CMU MULTIPIE 
database 


Raspberry Pi: 

The main hardware used in this project is Raspberry Pi 3 
model B. It is a Linux based platform which uses Python as 
programming language. In Linux software development is 
quite simple as it is an open source code development 
environment. 



Camera Module: 

The camera module used in this project is 8MP high 
definition camera for better picture quality. 



Conclusion 

Facial Expression recognition has wide application areas and 
it needs additional correct and reliable FER system. This 
work has given a survey on face expression recognition 


techniques. The feature extraction ways area unit exploited 
together with comparison, in line with the survey varied 
approaches area unit out there for establish face expression, 
during this paper numerous face expression recognition 
techniques, its associated areas and numerous analysis 
challenges is being surveyed. Methods with higher 
recognition rate have sensible performance These 
techniques give a temporal answer to the face expression 
recognition downside, feeling detection from face expression 
is a problem that causes difficulties. So, analysis during this 
space can continue for years as several solutions needs to be 
to make a perfect programmed. A comparative study is 
completed with numerous image information bases by 
differentiating the options. 
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